Existing HIV-1 immunogens are unable to elicit broadly reactive neutralizing antibodies (NAb) that are likely necessary for an optimally effective vaccine. Rare neutralizing monoclonal antibodies (mAbs) targeting the envelope glycoprotein exist and uncommon sera from HIV-1-infected individuals can neutralize diverse HIV-1 strains. From a well characterized patient cohort, we identified several sera displaying potent and broad HIV-1 neutralization and mapped the neutralizing specificity in the two broadest sera. Adsorption with gp120 derived from a single clade B isolate diminished serum neutralizing activity against cross-clade viral strains. A gp120 CD4 binding defective point mutant displayed substantially reduced adsorption capacity. For both sera, selective antibody elution and adsorption revealed a fraction of antibodies directed against the CD4 receptor binding site (CD4bs) that were able to neutralize diverse HIV-1 strains. Previously unidentified antibodies to the CD4bs exist in some HIV-1 sera, suggesting that new approaches to present this conserved region of gp120 to the immune system may result in vaccine immunogens that elicit broadly NAbs.